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ABSTRACT 

Two  problems  in  statistical  communication  theory,  the  detection  and  estimation  of  intel¬ 
ligence-carrying  signals  are  considered  for  the  case  of  transmission  through  a  multidimen¬ 
sional  Gaussian  random  channel.  Such  a  channel  is  characterized  by  the  property  that  a 
deterministic  input  results  in  a  set  of  received  waveforms  which  are  sample  functions  of 
Gaussian  processes. 

For  the  detection  problem,  a  receiver  is  found  which  operates  on  the  set  of  received 
waveforms  and  gives  as  outputs,  voltages  proportional  to  the  logarithm  of  the  likelihood 
functions  of  the  possible  transmitted  signals.  For  the  estimation  problem,  it  is  assumed 
that  the  intelligence-carrying  signal  is  itself  a  sample  function  of  a  Gaussian  process  and 
a  mathematical  description  is  presented  of  a  receiver  which  has  as  its  output  the  maximum 
a  posteriori  estimate  of  this  signal.  Examples  are  presented  in  which  optimum  receivers 
are  found  for  both  the  detection  and  estimation  problem. 
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ON  THE  DETECTION  AND  ESTIMATION  PROBLEM 
FOR  MULTIDIMENSIONAL  GAUSSIAN  RANDOM  CHANNELS* 
INTRODUCTION 

In  communication  systems  where  information  is  to  be  transmitted  from  a  source  to  a  re¬ 
ceiver,  the  intelligence-carrying  signal  is  always  partly  distorted  and  masked  in  transmission. 
A  major  problem  in  the  field  of  statistical  communication  theory  is  to  determine  a  method  to 
process  the  received  waveforms  in  order  to  recover  from  them  as  much  of  the  intelligence  as 
possible. 

In  certain  communication  systems  the  receiver  must  determine  if  a  certain  signal  (or  more 
generally,  which  of  given  set  of  signals)  is  present  in  a  received  waveform.  An  example  of  a 
system  of  this  type  is  a  digital  transmission  system  where  a  predetermined  set  of  signals  is 
chosen  as  the  signaling  alphabet.  In  other  communication  systems  the  receiver  is  required 
to  operate  on  the  received  waveforms  in  order  to  produce  an  estimate  of  the  waveshape  of 
the  intelligence-carrying  signal.  An  analogue  voice  communication  circuit  might  serve  as  an 
example  of  such  a  system.  The  processing  of  the  received  waveforms  in  these  two  types  of 
systems  will  be  called,  respectively,  detection  and  estimation. 

Much  of  the  original  work  on  the  detection^  and  estimation5®  of  signals  has  been  con¬ 
cerned  with  the  processing  of  a  single  received  waveform  which  consists  of  the  intelligence- 
carrying  signal  perturbed  by  additive  Gaussian  noise.  A  wide  variety  of  practical  problems, 
however,  cannot  be  satisfactorily  described  by  such  a  model.  In  the  first  place,  for  most 
practical  situations,  distortions  other  than  additive  Gaussian  noise  are  present  in  the  chan¬ 
nel.  Secondly,  in  many  communication  systems,  the  receiver  is  required  to  simultaneously 
process  multiple  received  waveforms  rather  than  the  single  received  waveform  assumed  in 
the  simplified  model.  In  this  report,  an  attempt  will  be  made  to  increase  the  generality  of 
the  mathematical  model  so  as  to  broaden  the  class  of  problems  to  which  the  theory  applies. 
For  other  treatments  of  this  problem,  the  reader  is  referred  to  the  work  of  Turin,9-10 
Price,11'12  Kailath13  and  others.14-15  The  portions  of  this  study  concerned  with  the  detec¬ 
tion  problem  will  be  found  to  overlap,  to  some  degree,  the  work  of  Kailath. 

FORMULATION 

In  this  report,  the  problems  of  the  detection  and  estimation  of  signals  will  be  considered 
for  the  situation  where  the  signals  have  been  transmitted  through  a  multidimensional 
Gaussian  random  channel.  For  the  purposes  of  this  report,  a  multidimensional  Gaussian  ran¬ 
dom  channel  will  be  defined  as  a  transmission  channel  which,  having  been  excited  by  any 
deterministic  signal,  results  in  a  set  of  received  waveforms  which  are  sample  functions  of 
Gaussian  random  processes.  Thus,  the  term  multidimensional  refers  to  the  multiplicity  of  the 

received  waveforms  and  the  term  Gaussian  describes  the  joint  statistics  of  these  waveforms 
‘Released  21  August  1961. 
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Figure  1.  Multidimensional  Transmission  Channel 

In  general,  these  waveforms  may  be  correlated  with  one  another.  It  is  to  be  noted  that  the 
channel  is  not  required  to  be  linear. 

Such  a  channel  can  be  interpreted  in  terms  of  the  block  diagram  shown  in  Figure  L 

Here,  the  intelligence-carrying  signal,  S(t),  enters  the  multidimensional  transmission 

channel  and  results  in  the  set  of  M  received  waveforms  r  (*)t  r  (j);  ...  ;  r  (t).  The  multi- 

12  M 

dimensional  channel  will  be  said  to  be  Gaussian  if  the  conditional  probability  of  this  set 
of  received  waveforms,  given  a  particular  S(t),  is  a  jointly  Gaussian  probability  density 
function. 

It  should  be  emphasized  that  the  ensemble  statistics  and  not  the  time  statistics  of 
the  received  waveforms  are  required  to  be  Gaussian.  For  instance,  in  the  case  of  a  single 
received  waveform  composed  of  a  deterministic  signal  in  additive  Gaussian  noise,  the  en¬ 
semble  statistics  of  the  received  waveform  are  Gaussian  while  in  most  cases  the  time 
statistics  are  not.  Thus,  the  classical  problem  of  the  detection  of  a  known  signal  in  addi¬ 
tive  Gaussian  noise  is  seen  to  be  a  special  case  of  this  model. 

Since  there  are  inherent  differences  in  the  formulation  of  the  detection  and  estimation 
problems,  it  will  be  convenient  to  discuss  each  of  these  problems  separately.  For  the 
detection  problem,  referring  to  Figure  2,  it  is  assumed  that  a  set  of  M  received  waveforms 

r  (0,  r  (<),  •••  )  r  (t),  are  available  at  the  receiver  over  an  interval  (0,  T).  These  wave 
12  M  .  (i ) 

forms  are  the  result  of  one  out  of  N  signals  S  ' (t),  i  =  1,2 ,  ,  /V,  chosen  by  the 


Figure  2.  Detection  Problem 
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transmitter  with  a  priori  probability  entering  the  transmission  channel.  The  M  re¬ 

ceived  waveforms  are  to  be  processed  by  the  receiver  so  as  to  determine  which  of  the  N 
signals  was  actually  chosen  by  the  transmitter.  A  complete  statistical  description  of  the 
set  of  received  waveforms  is  assumed  available  to  the  receiver  as  well  as  the  a  priori 
probabilities  of  the  transmitted  signals. 

A  slight  variation  of  this  detection  problem  which  arises  in  radar  applications  is  shown 
in  Figure  3.  Here  the  transmitter  always  transmits  the  same  deterministic  signal  but  the 


Figure  3.  Detection  Problem  for  Radar 

signal  travels  through  one  of  N  transmission  channels  with  some  given  a  priori  probability. 
These  N  channels  have  different  transmission  characteristics;  for  example,  in  the  case  of 
range  radar,  different  delays  would  be  assigned  to  the  different  channels.  The  receiver  is 
to  decide  on  the  basis  of  a  (0,  T)  sample  of  the  M  received  waveforms  r  (t) ,  r  ( t ), ... ,  r^t), 
through  which  of  the  /V  channels  the  signal  passed.  As  in  the  previous  case,  it  is  assumed 
that  the  statistics  of  the  received  waveforms  and  the  a  priori  probabilities  of  the  channels 
are  known  to  the  receiver.* 

For  either  configuration  of  the  detection  problem,  the  situation  is  one  of  multiple-alter¬ 
native  hypothesis  testing.  That  is,  given  the  set  of  received  waveforms,  the  receiver  is 
to  choose  among  N  mutually  exclusive  hypotheses.  For  the  radar  problem,  the  ith  hypoth¬ 
esis,  denoted  H corresponds  to  the  situation  where  the  signal  was  transmitted  through 
the  ith  channel,  while  for  the  communications  problem,  H  indicates  that  S  ^(t)  was  sent  by 
the  transmitter. 

A  short  discussion  of  multiple-alternative  hypothesis  testing  follows.  The  reader  is  re¬ 
ferred  to  the  references  for  amplification  of  these  remarks.  Woodward1  has  observed  that 
a  receiver  which  presents  tne  a  posteriori  probabilities  of  the  hypotheses  H i  =  1,2, ... ,  N, 

tells  as  much  as  it  is  possible  to  know  about  these  hypotheses  from  a  knowledge  of  the 

1 6-1 7  ° 

received  waveforms.  A  Bayes’  decision  rule,  which  minimizes  the  average  cost  of 

*  Because  of  the  repetitive  nature  of  the  radar  transmitted  pulse,  the  a  priori  probabilities  for  the  hypotheses 
during  the  interval  (0,  T)  can  sometimes  be  taken  as  the  a  posteriori  probabilities  for  the  hypotheses  during 
the  preceding  interval.  1 
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the  decisions,  has  been  shown  to  depend  on  the  relative  magnitudes  of  these  a  posteriori 

..18* 

probabilities.  Another  decision  rule,  the  Neyman-Pearson  criterion,  is  useful  in  testing 
between  two  hypotheses  where  it  is  not  convenient  to  assign  a  priori  probabilities  to  these 
hypotheses.  A  generalization  of  this  criterion  to  multiple-alternative  hypothesis  testing 
Has  shown14  that  the  decision  can  be  based  on  the  relative  magnitudes  of  the  likelihood 
functions*  for  the  hypotheses.  If  the  a  priori  probabilities  of  these  hypotheses  are  known, 
the  a  posteriori  probabilities  can  be  obtained  from  the  likelihood  functions  by  Bayes’  rule. 
Thus,  all  of  the  above  criteria  for  multiple-alternative  hypothesis  testing  depend  on  the 
relative  magnitudes  of  the  likelihood  functions  or  the  a  posteriori  probabilities  (which  can 
be  obtained  from  the  likelihood  functions).  Since  the  logarithm  is  a  monotonic  increasing 
function  of  its  argument,  decisions  can  be  made  using  any  of  these  criteria  on  the  basis 
of  quantities  which  are  proportional  to  the  logarithm  of  the  A  likelihood  functions. 

The  optimum  detector  can  then  be  divided  into  two  parts.  The  first  part  operates  on  the 
received  waveforms  and  results  in  outputs  which  are  proportional  to  the  logarithm  of  the 
likelihood  functions.  The  second  part  weights  these  outputs  in  accordance  with  some  cri¬ 
terion  in  order  to  make  a  decision  between  the  hypotheses.  In  this  report,  systems  will  be 
considered  only  for  instrumenting  the  first  part  of  this  detector. 

The  formulation  of  the  estimation  problem  will  next  be  considered.  It  is  assumed  that 
the  intelligence-carrying  signal  to  be  estimated  is  itself  a  sample  function  of  a  Gaussian 
random  process.  This  signal  enters  the  transmission  channel,  resulting  in  the  M  received 
waveforms  r  ^(t),  r  ^  (/),  ...  ,  r^(t).  This  problem  can  be  interpreted  in  terms  of  the  block 
diagram  shown  in  Figure  1,  but  now  S(t)  is  one  out  of  an  infinite  number  of  signals,  these 
signals  being  sample  functions  ot  a  Gaussian  process.  It  is  desired  to  operate  on  the  set 
of  received  waveforms,  received  during  the  interval  (0,  T),  to  obtain  an  optimum  estimate 
of  the  signal  S(t).  The  only  restriction  placed  on  the  channel  is  that  for  any  particular 
realization  of  the  signal,  the  set  of  received  waveforms  are  sample  functions  of  Gaussian 
random  processes;  that  is,  we  have  a  multidimensional  Gaussian  random  channel.  For 
simplicity  it  is  assumed  that  the  intelligence-carrying  signal  is  statistically  independent 
of  the  statistics  of  the  channel. 

It  should  be  noted  that  many  operations  normally  associated  with  the  transmitter  may  be 
considered  as  part  of  this  Gaussian  random  channel.  For  instance,  if  the  intelligence- 
carrying  signal  is  modulated  before  transmission,  this  modulating  process  (linear  or  non¬ 
linear)  can  be  included  in  the  channel  operation.  Thus,  the  estimation  of  modulated  signals 
corrupted  by  additive  Gaussian  noise,,  a  problem  previously  considered  by  Youla,^  would  be 
a  special  case  of  this  problem. 

The  optimization  criterion  used  in  this  report  is  that  of  maximum  a  posteriori  probability.5 
This  method  of  estimation  requires  the  receiver  to  choose  which  realization  of  the  intelli¬ 
gence-carrying  signal  was  most  likely  sent,  given  the  set  of  received  waveforms 

r  ( t ),  r  (i),  ...  ,  r  At).  More  explicitly,  the  receiver  is  to  choose  the  realization  of  the  intel- 
,12  M  , 

ligence-carrying  signal  which  maximizes  the  a  posteriori  probability.  This  optimization 

*  The  term  likelihood  function  is  used  to  describe  the  conditional  probability  of  the  effect  (the  received  wave¬ 
forms)  given  the  cause  (the  transmitted  signal)! 
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criterion  has  also  been  referred  to  in  the  literature  as  maximum  likelihood  estimation^  and 
a  posteriori  most  probable  estimation.® 

It  should  be  realized  that  the  output  of  the  receiver  in  the  estimation  problem  is  an 
analogue  signal  while  the  receiver  for  the  detection  problem  has  outputs  which  are  volt¬ 
ages  that  represent  the  conditional  probabilities  for  each  of  the  hypotheses. 

A  MULTIDIMENSIONAL  ORTHOGONAL  EXPANSION 


Since  both  the  detection  and  estimation  problems  have  been  phrased  in  terms  of  multiple 
received  waveforms,  it  is  convenient  to  introduce  vector  and  matrix  notation.  Consider  the 
M  waveforms,  n  ( t ),  n  2(t ),...,  n M{t ),  each  of  which  is  a  sample  function  of  a  random  pro¬ 
cess.  In  general  these  waveforms  may  be  correlated  with  one  another.  These  M  waveforms 
can  be  denoted  by  the  M  dimensional  vector the  / th  component  of  this  vector,  written 
representing  the  /  th  waveform.  Let  the  ensemble  means  of  these  waveforms  be  writ¬ 
ten  as  the  vector  m (r).  The  /  th  component  of  this  vector  mean  is  then  given  by  the  expres¬ 
sion, 

(1) 

where  E  {  }  denotes  the  statistical  expectation  of  the  term  in  the  brackets. 

The  second  order  statistics  of  these  M  received  waveforms  can  be  described  by  a  co- 
variance  function  matrix,  written  R(t,x).  The  element  in  the  /  th  row  and  A  th  column  of  this 
matrix  is  defined  by  the  expression 


=  E  ^  [r/(0  '  mj(t)  ]  trA(ac) - ]  }.  (2) 

Note  that  if  the  waveforms  are  sample  functions  of  Gaussian  random  processes  the  mean 
vector  m(t)  and  the  covariance  function  matrix  K  (t,*)  give  a  complete  statistical  descrip¬ 
tion  of  the  waveforms. 

In  expressions  involving  vectors  and  matrices,  the  following  notation  will  be  used.  The 
dot  product  of  two  M  dimensional  vectors  A  ( t )  and  written  A  ( t )  -B  (t),  is  given  by  the 
expression 

_  A  U 

A  (t)  •  B  (t )  _  2  A  .(t)  B ,.(?)•■  (3) 

i  =  i 

The  trace  of  the  M  x  M  matrix  R  (t,x)  will  be  written  T  r  [/?(/,*)  ]  and  is  defined  as 

'V  r\y 

A  M 

Tr  [R(t,*)]=  (4) 

Using  this  notation,  a  summary  will  be  presented  of  a  multidimensional  expansion  that  is 
useful  in  the  solution  of  both  the  estimation  and  detection  problems.  This  expansion  is  a 
generalization19  of  the  Karhunen-Loeve  20-2 1  expansion  to  vector  stochastic  processes.  The 
proof  of  the  following  statements  regarding  this  expansion  can  be  found  in  the  literature.14'15 

Let  the  M-component  vector  stochastic  process  ~n(t)  with  mean  vector  m(t)  and  covariance 
function  matrix  R(t,*)  be  represented  in  the  interval  (0,  T)  by  the  expression 
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n(t)  =  m(t)  +  2  N.  6.  (t)  ,  0  <  t  <  T. 

i  =  1  1  ' 

If  the  vectors  d.(t)  are  chosen  as  the  eigenvectors  of  the  matrix  integral  equation 


<5V 


0.(t)  =  K  f  R(t,x)  9  (x)  dx,  0  £  t  <_  T, 
I  l  o  ~  / 

and  if  these  vectors  are  orthogonal  and  normalized  such  that 


(6) 


fa.  (t)  • &ku)dt  -  s.k , 

then  the  coefficients  /V.  satisfy  the  following  equations: 


(7) 


E(N  }  =o, 

5.. 

E<N.n.  }  =.  — 
)  k  k 


(8) 

(9) 


Here  S  ,  is  the  familiar  Kronecker  delta  defined  as 

ik 


§  A  J  0  j1=k 
ik  |  1  j  —  k 

From  equations  (5)  and  (7),  the  coefficients  N-  are  given  by  the  expression 

T  _ 

N  =  f  [n(t)-m(r)l  •  9 At  )  d  t. 

I  »  / 


(10) 


(11) 


Note  that  subscripts  have  been  used  both  for  indexing  eigenvectors  of  integral  equations  and 
also  for  denoting  components  of  vectors.  Then,  the  k  th  component  of  the  vector  £?.(<)  will 
be  written#.  Jit)  ,  the  comma  between  the  subscripts  differentiating  this  notation  from  the 
double  subscript  notation  used  for  elements  of  matrices. 

From  a  generalization  of  Mercer’s  theorem,  the  i-  jth  element  of  the  covariance  function 
matrix  can  be  written 


;  V> 


(12) 


If  an  inverse  covariance  function  matrix,  Q  ( x,z ),  is  defined  as 


f  R  (t, x)  Q  (*, z)  dx  =  8  It-z  )  1 , 


(13) 


d 


where  1  is  the  unit  matrix  and  S(t-z)  is  the  Dirac  delta-function,  the  t - /th  element  of  this 
matrix  can  be  expanded  in  the  form 


Qij(x,z) 


(14). 


Let  us  now  assume  that  the  components  of  n(t)  are  sample  functions  of  Gaussian  random 
processes.  Then,  the  coefficients  /V.  are  Gaussian  random  variables  and  from  equation  (9) 
are  uncorrelated  and  thus  statistically  independent.  The  probability  density  for  the  noise 
vector  ~n{t),  which  is  the  joint  probability  of  the  set  of  coefficients  N .  ,  can  then  be  writ¬ 
ten* 


pOO  =  n 

k  =1 


A 

/  277- 


exp 


'  \  \  < 


or 


p(/«  )  -  n 

k=\ 


I  1  0. 

jexpj-  -  2  k.N  ! 

J  T  2  A=i  *  *  1 


(15) 


(16) 


From  equation  (11 ),  the  series 


I 
k  - 1 


A 


can  be  written 


®  T  T  M  M  oo 

'k2  -  f  fit  (  nt(l)  -  mi  (<)■}  [n  j(x)  -  m  j(x)]  ^  ^8 ^  ,(t)  8 ^  d  t  d  x.  (17) 

Referring  to  equation  (14),  and  using  vector  and  matrix  notation,  this  series  becomes 


T  T 


1  k,N  2  =  f  f  (*)  *  m  (*)]  *  ln(x)  -  m(x)]  dt  dx. 

k= l  k  k  Q  Q 

Since  Q^U.x)  =  Q^(x,t)  it  can  be  shown  that  equation  (18)  can  be  rewritten  as 


(18) 


OD  T  t 

1  k  N  ^  -  2  j[n(t)-m  (t) ]  •  [  f  Q(t,x)  [n(x)-m(x)]  d  xJ  d  t.  (19) 

k= l  *  0  o  ~ 

From  equation  (16),  the  natural  logarithm  of  the  probability  density  for  the  noise  vec¬ 
tor  H(t)  can  be  expressed  as 


log  p(n)  =  K  +  ^  2  log  A.  -  |  2  A.  N k2 , 
2  k=  i  k=i 


where  K  is  a  constant.  Combining  equations  (19)  and  (20)  we  can  write 
1  ®  T  t 

log  p(n)  —K  +~  2  log  k -  J  [r»(t)  -  m(t)]  •  [  f  Q(t,x)  ln(x)  -  m (x) ]  dx]  it. 
1  *=i  *  o  o  ~ 


(20) 


(21) 


*  It  is  realized  that  some  difficulties  are  encountered  in  the  convergence  of  equations  (15)  and  (16).  Such 
difficulties  can  be  avoided  by  using  a  finite  number  of  terms  in  the  series  or  by  considering  the  ratio 

where  n  n(t)  is  a  reference  noise  process.  These  difficulties  will  be  ignored  in  this  report  but  the 

reader  should  understand  that  one  of  these  two  procedures  can  always  be  used  to  rigorize  the  work  to 
follow. 
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In  the  work  to  follow,  the  set  of  received  waveforms  r  ( t )  will  be  represented  in  the  interval 
(0,7')  by  a  multidimensional  orthogonal  expansion  with  uncorrelated  coefficients  similar 
to  that  given  for  ?r(t)  and  described  in  equations  (5)  through  (11).  In  the  most  general  case> 
the  mean  vector  and  the  covariance  function  matrix  may  depend  upon  which  particular  real¬ 
ization  of  the  intelligence-carrying  signal  was  transmitted  (for  the  estimation  problem)  or 
which  hypothesis  was  true  (for  the  detection  problem).  Note  that  if  the  covariance  function 
matrix  depends  upon  these  conditions  so  will  the  eigenvectors  and  eigenvalues  of  the  ma¬ 
trix  integral  equation  having  the  covariance  function  matrix  as  its  kernel.  The  superscript 
i  will  be  used  with  functions  in  the  detection  problem  to  indicate  that  hypothesis  H.  Was 
assumed  true  in  writing  those  functions.  For  the  estimation  problem,  the  superscript  s 
on  the  functions  will  indicate  that  a  particular  realization  of  the  intelligence-carrying  sig¬ 
nal  was  assumed. 

CALCULATION  OF  THE  LIKELIHOOD  FUNCTIONS  (DETECTION  PROBLEM) 

In  this  section  systems  will  be  derived  which  operate  on  the  s.;t  of  received  waveforms 
and  result  in  outputs  which  are  proportional  to  the  logarithm  of  the  /V  likelihood  functions 
corresponding  to  the  /V  possible  hypotheses.  The  mathematics  needed  to  derive  expres¬ 
sions  describing  these  systems  have  been  discussed  in  the  previous  section.  By  analogy 
with  this  work,  let  us  consider  that  the  set  of  received  waveforms ,  T(t),  are  expanded 
in  a  multidimensional  orthogonal  expansion,  under  the  assumption  that  hypotheses  H .  is 
correct.  Then,  following  the  development  given  in  the  last  section,  an  equation  is  obtained 
for  the  logarithm  of  the  conditional  probability  density  of  7 (t),  assuming  the  truth  of  hypoth¬ 
esis  H . .  This  conditional  probability  density  is  just  the  likelihood  function  for  hypoth¬ 
esis  H .  and,  referring  to  equation  (21),  is  given  by  the  expression 

00  T  t 

log  p(r|  f/.)  =  K  +  1  2  log  X  (l)  -  J  fr(t)  -  m  («)]  •  [  f  Q(,y (t,x)  [T(x)  -  m  (°(*)]  dx]  dt. 

‘  1  *=l  *  o  o 

(22) 

Here,  m*‘^(t)  is  the  vector  mean  of  r(t),  assuming  hypotheses  H ^  (t,x)  is  the  inverse 

covariance  function  matrix  of  r(t),  assuming  hypothesis  H ....  are  the  eigen¬ 
values  of  a  matrix  integral  equation  having  the  covariance  function  matrix  R^\t,x)  as  its 
kernel;  and  K  is  a  constant. 

To  emphasize  the  meaning  of  the  notation,  the  jt h  component  of  the  vector  mean 
m  ■  (t)  is  given  by  the  expression 

m.(‘>(r)  =  E  { ry(0  1 77 (23) 

where  E  {  r.(t )  | H ^  is  the  conditional  expectation  of  r.(t),  assuming  hypothesis  H ^  Also 
the  /-&  th  element  of  the  covariance  function  matrix  R^‘\t,x)  is  given  as 

Rp(t,x)  ■  ^.(O-mj*^)]  lrh'x)  -  m  «>(*)]  \H .}.  (24) 
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The  receiver  is  then  to  calculate  the  functions  log  p(T  |  H f)  for  i  =  1,  2,  ...  ,  N.  The 
assumption  has  been  made  that  the  receiver  has  a  complete  statistical  description  of  the  set 
of  received  waveforms.  Since  the  received  waveforms  are  Gaussian  random  processes,  the 
receiver  must  have  available  the  components  of  the  vector  mean  and  the  elements 

of  covariance  function  matrix  R  (i)(t,x)  for  all  values  of  i. 

As  an  illustrative  example,  consider  the  transmission  channel  shown  in  Figure  4  where 
the  additive  noise  n^i)  and  multiplicative  noise  «2(t)  are  Gaussian,  have  zero  mean  and 
are  uncorrelated  with  one  another.  Then  the  received  waveforms  are  Gaussian  processes 


Figure  4.  Example 

with  means  and  variances  given  by  the  equations; 


=  sU)(‘) 

(25) 

m£t)  -  0 

(26) 

(27) 

«2«>(t,*)  =  E  { n  2(l)  n  2(x)}  s(i)(t )  S<%) 

(28) 

Rxf  (*,*>  =  *2(jW)  •Einl(tUi(x))  S<%). 

(29) 

The  receiver  must  then  have  knowledge  of  the  N  signals  S(1)(t),  ...  ,  and 

the  covariances  E  in^t)  n^*)},  E  {n2(t)  n2(*)},  and  E  {«■  ■(*)• »2(*)}. 

Let  us  now  return  to  equation  (22)  and  investigate  systems  which  will  calculate  these 
functions.  Defining  the  quantities  8(,)  and  C(,)(F)  as 
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(30) 


•m  -j 

C {i )(y)  =  -  j  [ 7(1)  -  m  <*' >  (t)]  •  [/  <? (i > (i,*)  [  r(*)  -  7n  (,)(*)]  <f*  ]  <fr,  (3 1 ) 

0  0  ~ 

we  can  ./rite 

log  p(r\H.)  =  a:  +fl(i)  +  C(,)(r).  (32) 

The  first  term,  the  constant  K,  is  independent  of  the  index  i  and  thus  can  be  ignored.  The 
second  term,  B^‘\  will  in  general  be  different  for  different  values  of  the  index  i.  However, 
it  does  not  represent  an  operation  on  the  received  waveforms  and  can  be  calculated  prior 
to  communicating.  It  should  be  noted  that  if  the  covariance  function  matrix  R^l\t,x)  is 
independent  of  the  index  t,  then  6(,)  would  also  be  independent  of  the  index  i  and  could 
also  be  ignored.  The  C(,)(r)  is  a  function  that  must  be  calculated  from  the  M  received 
waveforms.  Rewriting  equation  (31)  in  terms  of  the  components  and  elements  of  the  vectors 
and  matrices  we  have 

C(i)(r)  =  f  L  1  [r.  (t)  -  JQ  .[* '  (/,*)  irk(x)  -  mkU)(x}]  dxdt,  (33) 

o  j=\k  =  l  I  o’ 

wiich  explicitly  displays  the  operations  necessary  to  calculate  C  (i \t). 

A  system  for  calculating  the  function  log  p  (t  \H i  )  for  the  case  of  two  received  wave¬ 
forms  is  shown  in  Figure  5.  In  this  diagram,  the  blocks  labeled  Q ^  (t,x)  are  linear  filters, 
Qij(t,x)  being  the  response  at  time  t  of  the  filter  to  a  unit  impulse  applied  at  time  *.  The 
complete  receiver,  of  course,  must  contain  M  of  these  systems,  one  for  each  possible 
hypothesis.  A  complete  receiver  is  shown  in  Figure  6  for  the  single  input  case. 

A  receiver  which  has  outputs  which  are  proportional  to  the  a  posteriori  probabilities 
of  the  hypotheses  instead  of  the  likelihood  functions  can  be  obtained  by  a  slight  modifica¬ 
tion  of  the  previous  receiver.  From  Bayes’  rule,  the  a  posteriori  probability  of  H  written 
p(H i  |r),  is  given  in  terms  of  the  likelihood  function  p(r]H.)  and  the  a  priori  probability 
p  ( W.)  as 

p(tf,|.7)  =  k  p(tff),  (34) 

where  £  is  a  constant.  Taking  logarithms  and  referring  to  equation  (32),  it  is  seen  that 
the  logarithm  of  the  a  posteriori  probability  for  H .  can  be  written 

log  P(H.\r)  =  (K+h)  +  +  log  p  ( H .))  +  C«)( 7).  (35) 

Thus,  the  same  basic  receiver  can  be  used  to  obtain  the  functions  log  p(Hj  |r)  except 
for  the  bias  term  which  is  changed  by  the  addition  of  the  term  log  p(H .).  If  all  hypotheses 
have  equal  a  priori  probabilities,  no  modification  of  the  basic  receiver  is  required. 

Configurations  for  obtaining  the  likelihood  functions  (and  a  posteriori  probabilities) 
can  be  devised  other  than  the  one  described  in  equations  (30)  through  (33)  and  shown  in 
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Figure  5.  Colculotion  of  log  p(r  |//()  for  Two  Received  Waveforms 
Figures  (5)  and  (6).  For  instance,  log  p  (T \h  .)  can  be  written, 


log  P  (T\H.)  =  K  +  P<‘>  +£<*>(-), 

where  P(,)  a  bias  term  which  can  be  calculated  before  transmission,  is  defined  as 

T  T 

)  .  ± 

* 


DU)  =  \  1  log  A<*‘>  f  f  mU\t)  'QUH‘,Z)  r.  (iH*)  dx  dt 

L  lc- 1  *  ^ 


=1  "  “00 

and  a  term  which  is  calculated  from  the  received  waveforms,  is  given  as 

T  T 


A  r  rr^liiM  1^/ 

Note  that  if  the  inverse  covariance  function  matrix  is  independent  of  the  index  i. 


&i](T)  S  /  /[«<*>(*)  -  -7(t)]  *  Q(i\t,x)  T(x)  dx  dt. 
0  0  1  ~ 


(36) 


(37) 


(38) 
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Figure  6.  Complete  Receiver  -  Nondiversity  Cose 
T  T 

the  term  -  1/2  f  f  T(t)  •  Q(t,x)  T(x)  d  x  d  i  will  be  the  same  for  all  hypotheses  and  thus 
o  o  ~ 

can  be  neglected. 

An  example  illustrating  some  of  these  ideas  for  the  case  of  a  single  received  waveform 
is  shown  in  Figure  7  Here,  the  transmitter  chooses  one  of  the  signals  sin  ai.t,  i  =  1, 

2,  ...  /V,  and  transmits  it  during  the  interval  (0,  T).  The  multiplicative  noises  m  (*)  and 
m  (t)  are  assumed  to  be  uncorrelated,  stationary,  Gaussian  processes  with  zero  means 
and  identical  covariances  R  (t-x), 

m  jX*)  }  =  E  {m2(t)  m 2(x)  }.  (39) 

The  additive  noise  n(t),  which  is  uncorrelated  with  the  multiplicative  noises,  is  assumed 
to  be  a  stationary  Gaussian  process  with  zero  mean  and  covariance  Rn(t-x).  The  block 
denoted  90  degrees  is  assumed  to  be  a  90  degree  phase  shifter;  that  is,  a  nonphysically 
realizable  linear  filter  whose  output  is  the  Hilbert  transform  of  its  input. 
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Figure  7.  FSK  Transmission  through  a  Fading  Channel 

If  the  keying  interval  T  is  large  enough  (T  >  >  ^  for  all  i)  the  output  of  the  <?0  de- 

i 

gree  phase  shifter  will  be  cos  u.t  and  the  received  waveform  can  be  written 

r(t)  =  n(i)  +  m^t)  sin  o>(.t  +  n>2(t)  cos  cu.t,  (40) 

or 

r(t)  =  n(t)  +  R(t)  sin  [  a >.t  +  4>  (t)  ]  ,  (41) 

where  R(t)  is  Rayleigh  and  4>(t)  is  uniformly  distributed  (*  v,  n).  This  model,  which  has 
been  previously  considered  by  Price.  11  then  represents  a  narrow-band  Rayleigh  fading 
signal  with  additive  Gaussian  noise. 

The  mean  of  r(t)  is  identically  equal  to  zero  for  all  H .  and  the  covariance 
can  be  written  as  the  difference  of  the  two  variables  t  and  *  and  is  given  by  the  expres¬ 
sion 

R{l)(l-X )  =  Rn(t-x)  +  R'm(t-x)  cos  CU(.(£-*).  (42) 

Referring  to  equations  (30)  through  (32)  it  can  be  shown  that  the  bias  term  B(i)  is  the  same 
for  all  hypotheses.  The  optimum  receiver  is  then  given  by  the  block  diagram  shown  in  Fig¬ 
ure  8,  where  the  impulse  responses  A(,)(x,z)  of  the  time  varying  linear  filters  are  the  solu¬ 
tion  to  the  integral  equation, 


T 

f  C  R „(-*-*)  +  cos  &>;(<-*)]  hil)(x,z)  dx  =  5  (r-z),  (43) 

o 

0  <  t,  z  <  T. 

Ignoring  the  physical  realizability  of  these  filters  and  assuming  that  T  is  large  enough  so 
that  Fourier  transform  techniques  can  be  used,  the  transfer  function  of  the  ith  filter  is 
given  approximately  by  the  expression 
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Figure  8.  Optimum  Receiver  for  Example  Shown  in  Figure  7. 

Here  4>n(co)  and  <pm  (&>)  are  respectively  the  power  spectra  of  the  additive  and  multiplica¬ 
tive  noises.  Thus  the  filters  are  essentially  band  reject  filters  with  the  stop  band  cen¬ 
tered  at  <*).. 

I 

The  outputs  of  the  receiver  shown  in  Figure  8  are,  except  for  the  omission  of  additive 
constants,  the  logarithm  of  the  likelihood  functions  (or  the  logarithm  of  the  a  posteriori 
probabilities  since  the  a  priori  probabilities  were  assumed  equal).  These  functions  are 
then  used  in  making  the  decision. 

As  a  second  example  consider  the  transmission  system  shown  in  Figure  9,  where  the 
intelligence-carrying  signal  S(t)  is  one  of  the  N  signals  S(l)(t),  i  =  I,  2,  ...  ,  IV.  The  two 


m(t ) 


n,  (t) 

©- 


r  *  »(  t  ) 


<+> 


n2  ( 1 1 


(2)  v 
r  (t) 


Figure  9.  Transmitted  Reference  Communications  System 
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received  waveforms  are  given  as 


rU)(0  =  S{i\t)m(t)  +  n t (t)  0  <  i  <  T 

r(2>(  t  )  =  m(t) .  +  n2  (r)  . 


It  is  assumed  that  the  noises  »j(t),  /»2(/)  and  m  (t)  all  are  sample  functions  of  independent 
Gaussian  processes  with  zero  means  and  covariance  functions 


N.. 

E  (71,0)  n,(r)}  =  S(t-r),  (46) 

N02 

E  {  n  2(t)  n  2('T)}  =  $  (t  -  r),  (47) 

M 

£  {m(t)m(T)  }  =  8(/-r).  (48) 


Such  a  situation  would  occur  when  a  wide-band  Gaussian  noise  m  (t)  is  used  as  the  carrier 
for  transmitting  digital  signals  and  this  noisy  carrier  is  also  transmitted  over  another  chan¬ 
nel  for  use  by  the  receiver  in  the  detection  process. 

The  mean  vector  of7(t)  is  identically  equal  to  zero  and  the  covariance  matrix 
is  given  as 


K  <•)(<,*)  = 

~  2 


Mq  +  /VQ1  M0S«>(l) 


Mo+^oa. 


(49) 


The  inverse  covariance  function  matrix  Q  (l\t,x)  is  then 

"V. 


*o,*oi  +M0(lVo2S(i)(OS(‘)(*)  +/V01) 

"  Mo  +  \2 

:MtS<‘>(t)  M0S«>(*)S«>(*)  +  iV01_ 

To  further  restrict  the  problem  it  is  assumed  that  the  transmitter  is  sending  equi- 


probable  binary  digits,  the  two  possible  signals  being  a  signal  S(t)  and  its  negative, 
that  is, 


(50) 


s(i)(0  =• 


S(t) 


1=1 


(51) 


\-S(t)  i  =  2. 

In  this  case  it  can  be  shown  that  the  bias  term  is  the  same  for  both  hypotheses.  The 
logarithm  of  the  likelihood  function  is  then  given  as 


^Kx+  4M0  f  S(Qr(1)(t)  r(2)(Q  d  t _ 

°°  Vo2+W2^)+JVo.) 


t=  1 


log  p (r  |  H(  )  =  / 


,  (52) 


T  S(t)r{1)(t)  r<2)(t)  tit 


where  A  is  a  constant.  Thus  the  optimum  receiver  calculates  the  quantity 


i  =2 


T  S(t)rn)(t)  r(2>(0  dt 

Note  that  if  S(t)  is  a  constant  the  optimum  receiver  just  computes  the  short-time  cross¬ 
correlation  of  the  two  received  waveforms. 

MAXIMUM  A  POSTERIORI  PROBABILITY  RECEIVER  (ESTIMATION  PROBLEM) 

Let  us  now  assume  that  the  intelligence-carrying  signal  S(i)  is  a  sample  function  of 
a  Gaussian  random  process  with  zero  mean  and  covariance  Rs(t,x).  The  signal  is  trans¬ 
mitted  through  a  multidimensional  Gaussian  random  channel  and  the  statistics  of  the 
signal  and  the  channel  are  independent.  It  is  desired  to  operate  on  the  set  of  received 
waveforms,  T(t),  received  during  the  interval  (0,  T),  in  order  to  obtain  the  maximum 
posterior  estimate  of  the  signal  S(t).  That  is,  the  output  of  the  receiver  is  to  be  that 
particular  realization  of  S(t)  that  maximizes  the  a  posteriori  probability  p  (S  |r")  .  Since 

the  logarithm  is  a  monotonic  increasing  function  of  its  argument,  the  realization  of 
5(,0>  which  maximizes  the  a  posteriori  probability  will  also  maximize  the  logarithm  of 
the  a  posteriori  probability. 

By  analogy  with  the  work  on  detection,  let  us  assume  a  particular  realization  for  S(t) 
and  expand  the  set  of  received  waveforms  in  the  orthogonal  expansion 

7(0  -~m  <s)(0  +  X  £<s>(0,  0  <  K  T,  (53) 

/  =  1 

where  the  dSs  *  (0  are  eigenvectors  of  the  matrix  integral  equation 
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and  are  orthonormal;  that  is, 


T  _ 

I  9{s)(t)  •  0f<*>'(t)  it  =  8  (55) 

In  these  equations  ~m  (s ' (t)  is  the  vector  mean  and  fl(s>(/,x)  is  the  covariance  function 
matrix  of  the  received  waveforms,  both  assuming  a  particular  realization  of  S(t).  Then, 
the  logarithm  of  the  conditional  probability  of  7(t)  given  a  particular  S(r)  can  be  written 


log  p(T\S)  =  K  +  I  1  l0g  X<*>  -L  2  X(*>  (a<s>)2  .  (56) 

2  i  =i  7  1  i  =i  /  / 

Let  us  now  expand  the  signal  S(t)  in  an  ordinary  Karhunen-Lofeve  expansion.  That  is, 


let 


s  (t)  =IAW<), 

/  =i  7  7 


where  the  0  (t)  are  the  eigenfunctions  of  the  integral  equation 

/ 

y.  /  «s(t,*)  ^.(*)  <r*  =  0.(o,  o  <  <  <  r, 

and  are  orthonormal, 


/  'Pj  (0  </»A(0 

Then  the  logarithm  of  the  a  priori  probability  of  5(t)  can  be  written  as 

log  p(S)  =  Kt  +  \  l  log  y  -  \  l  y  /S'.2  . 

Z/=l  7  Z  '  ’ 


(57) 


(58) 


(59) 


(60) 


Using  Bayes’  rule  and  referring  to  equations  (57)  and  (60),  the  logarithm  of  the  a 
posteriori  probability  for  S{t)  can  be  written 

lo*  p<sln  ■  i  log  «>> 


The  signal  5(t)  which  maximizes  the  logarithm  of  the  a  posteriori  probability  will  be  found 
by  solving  for  the  set  of  coefficients  /3^  that  makes  equation  (61)  a  maximum.  Thus,  taking 
the  partial  derivative  of  the  terms  in  equation  (61)  with  respect  to  /3^  and  setting  the  re¬ 
sult  equal  to  zero  results  in  the  equation 


(62) 


i  i  _i_  ?£1  --I  A  al- [ Vs '(V* 1)81  ■y*4- 


2,~iV*’  3  ft 


2  3/3,  >  t 


Multiplying  equation  (64)  by  the  expression 


0*(O 

?A 


and  summing  over  the  index  A  results 


in  the  following  expression  for  the  estimate  of  S(t), 


,  00  00  .  3  X.  .  0,  (f) 

5  CO  =  J  2  ^  -r  — 

2  /  =  !  A=i  x  A  7, 


1 


*  *  *  0i;(O 

--ill  -A.  [  \  .  (  a  .)2} 

t  '*  2'*1*-1  3A  '  '  2* 


(63) 


The  notation  S  (t)  is  used  to  indicate  an  estimate  of  S(t)..  The  functions  with  the  super¬ 
script*  also  depend  upon  this  estimate. 

It  is  desired  to  find  more  convenient  forms  for  the  terms  on  the  right  hand  side  of  equa¬ 
tion  (63)-  It  is  shown  in  the  Appendix  that  the  first  term  can  be  written  as 
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and  the  second  term  becomes 
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Then  the  optimum  estimate  S  (t)  is  given  by  the  sum  of  these  terms. 

The  inverse  covariance  function  matrix  Q(y,x)  will,  in  general,  be  a  function  of  the 
intelligence-carrying  signal-  This  functional  dependence  may  occur  through  the  appearance 
of  terms  which  are  explicit  functions  of  S(x)  and  S(y).  However,  the  matrix  Q(y,x)  can  depend 

'V' 

implicitly  on  the  intelligence-carrying  signal;  for  example,  Q(y,x)  could  be  a  function  of 
T  A 

f  S2(z)d  z.  A  similar  situation  may  occur  for  the  vector  mean  m(x).  If  the  intelligence^ 


carrying  signal  appears  in  Q(y,x)  and  m  (x)  only  in  terms  which  are  explicit  functions  of 
S(x )  and  S(y),  then  it  is  shown  in  the  Appendix  that  equation  (64)  becomes 
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and  equation  (65)  can  be  written  as 
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For  this  case,  the  optimum  estimate  of  S(t)  is  given  by  the  expression 
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dy  dx. 


In  addition,  if  the  covariance  function  matrix  R(x,y)  is  independent  of  the  signal  transmitted, 
the  estimate  S  (t)  is  given  as 
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while  if  the  vector  mean  m(x)  is  identically  equal  to  zero  the  estimate  5  (t)  can  be  written 
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If  the  inverse  covariance  function  matrix  or  the  vector  mean  have  an  implicit  dependence 
on  the  intelligence-carrying  signal,  equations  (66)  through  (70)  do  not  apply  but  equations 
(63)  through  (65)  are  still  valid.  Expressions  similar  to  equations  (66)  through  (70)  can 
still  be  obtained  for  particular  problems  but  the  form  for  these  expressions  would  change 
from  problem  to  problem. 

As  an  example  of  an  estimation  problem,  again  consider  the  transmitted  reference  com¬ 
munications  system  shown  in  Figure  9-  It  is  assumed  that  the  intelligence-carrying  signal 
S(t)  is  a  sample  function  of  a  Gaussian  process  with  zero  mean  and  covariance  R s(t,x).  The 
additive  and  multiplicative  noises,  m(t),  n  ^(t)  and  n^O),  are  again  assumed  to  be  sample 
functions  of  independent  Gaussian  processes  with  zero  means  and  covariance  functions  as 
in  equations  (46)  through  (48). 

The  inverse  covariance  function  matrix  of  the  received  waveforms  assuming  a  particu¬ 
lar  realization  of  S(t)  is  then,  from  equation  (50),  given  as 
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Note  that  this  matrix  only  has  an  explicit  dependence  on  the  intelligence-carrying  signal  so  that 

3 Q\y,x ) 

equations  (66)  through  (70)  can  be  used.  The  quantity  — -  is  then  given  as 
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Equations  (73)  and  (74)  are  then  substituted  into  equation  (70)  to  obtain  an  expression  for 
S  ( t ).  It  is  to  be  remembered  that  all  terms  affixed  with  an  asterisk  depend  upon  the 
estimate  S*(t). 

SUMMARY 

Two  classical  problems  in  statistical  communication  theory,  the  detection  and  estima¬ 
tion  of  intelligence-carrying  signals,  have  been  considered  for  the  case  of  transmission 
through  a  multidimensional  Gaussian  random  channel.  This  channel  is  characterized  by 
the  property  that  a  deterministic  signal  results  in  a  set  of  Gaussian  (in  general,  correlated) 
received  waveforms. 

For  the  detection  problem,  a  receiver  was  found  which  operates  on  the  set  of  received 
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waveforms  and  gives  as  outputs, voltages  proportional  to  the  likelihood  functions  of  the 
possible  transmitted  signals.  Two  configurations  for  this  receiver  are  mathematically  de¬ 
scribed  in  equations  (30),  (31)  and  (32)  and  equations  (36),  (37)  and  (38).  A  modification 
to  this  receiver  which  results  in  outputs  which  are  proportional  to  the  a  posteriori  proba¬ 
bilities  of  the  signals  is  described  by  equation  (35).  This  theory  was  then  used  to  find 
the  optimum  detector  for  FSK  transmission  through  a  particular  type  of  fading  channel. 

The  optimum  detector  was  also  found  for  a  transmitted  reference  communications  system. 

For  the  estimation  problem,  a  receiver  was  found  which  has  as  its  output  a  signal 
which  is  the  maximum  a  posteriori  estimate  of  the  intelligence-carrying  signal.  Various 
forms  of  this  receiver  are  described  mathematically  in  equations  (63)  through  (70).  As  an 
example,  a  mathematical  expression  was  found  for  the  optimum  estimate  of  a  Gaussian 
intelligence-carrying  signal  for  the  case  of  a  transmitted  reference  communications  system. 
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APPENDIX 


In  this  Appendix,  equations  (64)  through  (67)  of  the  text  will  be  derived.  The  notation 
of  this  Appendix,  except  for  the  omission  of  the  superscripts*,  will  be  identical  to  that  of 
the  text  and  use  will  be  made  of  previously  defined  functions. 

Referring  to  the  definition  of  the  trace  of  a  matrix  given  in  equation  (4),  and  using  the 
expansions  for  ft  (x ,  y )  and  Q(x,y)  given  in  equations  (12)  and  (14),  we  can  write, 
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Noting  the  orthonormality  of  the  vectors  6  I*  ),  this  equation  becomes 
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The  last  two  expressions  on  the  right  hand  side  of  equation  (76)  are  zero  as  can  be  seen  by 
writing 
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Multiplying  the  remaining  terms  in  equation  (76)  by 


and  summing  over  the 
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index  k  results  in  the  expression 
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which  is  equation  (64)  of  the  text. 

If  @  (*,y  )  depends  on  the  intelligence-carrying  signal  (and  thus  on  /3, )  only  through 
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so  that 
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Since  £>  Gt,y  )  =  Q  (y,*  )  and 
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we  can  finally  write 
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which  is  equation  (66)  of  the  text. 

From  equations  (53)  and  (55),  we  can  write 
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Taking  the  partial  derivative  of  both  sides  of  equation  (84)  with  respect  to  /3  multiplying  by 
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and  summing  over  the  index  k  results  in  the  equation 
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which  is  equation  (65)  of  the  text. 

Let  us  restrict  ourselves  to  the  case  where  Q(y,x)  and  m  ( x )  depend  on  the  intelligence- 

"V 

carrying  signal  only  through  implicit  functions  of  S(x)  and  S(y). 

Then  equation  (85)  can  be  rewritten  as 
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Referring  to  equation  (82),  equation  (86)  becomes 
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